Services are being developed which take real world observational data as inputs. Examples of such observational data include local weather, the numbers of people in particular locations, the movement of those people and in some cases, the identity of those people. It is hard to test these services because the real world observational data is required.
One testing solution is to use a pilot environment, however, there is considerable management overhead associated with setting up and controlling scenarios which are performed in this pilot environment and also it is difficult to capture what actually happened during the scenarios. The logistics of setting up the pilot environment, organizing people to performed scripted scenarios etc can also be problematic. It is also difficult to replay scenarios in a pilot environment because the people may not act in exactly the same manner twice.
Another testing solution is to use a proprietary simulator. These simulators are purpose built for particular scenarios and are generally restricted in terms of their visual representation of the scenarios. It is expensive, complicated and time-consuming to set up these proprietary simulators which must be re-written for new scenarios.